home *** CD-ROM | disk | FTP | other *** search
- Subject: v19i078: Cnews production release, Part01/19
- Newsgroups: comp.sources.unix
- Sender: sources
- Approved: rsalz@uunet.UU.NET
-
- Submitted-by: utzoo!henry
- Posting-number: Volume 19, Issue 78
- Archive-name: cnews2/part01
-
- [ This is a rewrite of the transport and expire portions of the Usenet
- software. It was designed to be fast and flexible. --r$ ]
-
- : ---CUT HERE---
- mkdir doc
- mkdir conf
- mkdir notebook
- mkdir batch
- mkdir contrib
- mkdir contrib/nntpmail
- mkdir contrib/nntpmail/post_via_mail
- mkdir contrib/nntpmail/mailing_lists
- mkdir contrib/nntpmail/mailing_lists/appendfile
- mkdir contrib/nntpmail/nntp_support
- mkdir contrib/nntpmail/mail_to_group
- mkdir contrib/rn.mod
- mkdir expire
- mkdir h
- mkdir hfake
- mkdir input
- mkdir libbig
- mkdir libbsd42
- mkdir libc
- mkdir libcnews
- mkdir libfake
- mkdir libsmall
- mkdir libstdio
- mkdir libusg
- mkdir libv7
- mkdir libv8
- mkdir man
- mkdir misc
- mkdir nntpdiffs
- mkdir nntpdiffs/diff
- mkdir nntpdiffs/src
- mkdir nntpdiffs/src.allnew
- mkdir relay
- mkdir relay/regress
- mkdir relay/regress/out
- mkdir relay/regress/out/test
- mkdir relay/regress/out/test/a
- mkdir relay/regress/out/test/b
- mkdir relay/regress/out/test/c
- mkdir relay/regress/out/control
- mkdir relay/regress/master
- mkdir relay/ads
- mkdir relay/anews
- mkdir relay/ctl
- mkdir relay/aux
- mkdir relay/altctl
- mkdir relay/sh
- mkdir rna
- mkdir rna/lib
- echo 'COPYRIGHT':
- sed 's/^X//' >'COPYRIGHT' <<'!'
- X/*
- X * Copyright (c) University of Toronto 1985, 1986, 1987, 1988, 1989.
- X * All rights reserved.
- X * Written mostly by Geoffrey Collyer and Henry Spencer.
- X * This software is not subject to any license of the American Telephone
- X * and Telegraph Company or of the Regents of the University of California.
- X *
- X * Permission is granted to anyone to use this software for any purpose on
- X * any computer system, and to alter it and redistribute it freely, subject
- X * to the following restrictions:
- X *
- X * 1. The authors are not responsible for the consequences of use of this
- X * software, no matter how awful, even if they arise from flaws in it.
- X *
- X * 2. The origin of this software must not be misrepresented, either by
- X * explicit claim or by omission. Since few users ever read sources,
- X * credits must appear in the documentation.
- X *
- X * 3. Altered versions must be plainly marked as such, and must not be
- X * misrepresented as being the original software. Since few users
- X * ever read sources, credits must appear in the documentation.
- X *
- X * 4. This notice may not be removed or altered.
- X */
- !
- echo 'PATCHDATES':
- sed 's/^X//' >'PATCHDATES' <<'!'
- X23-Jun-1989
- !
- echo 'README':
- sed 's/^X//' >'README' <<'!'
- XThis is C News, superseding assorted preliminary releases. 9 June 1989
- X
- XC News is a reimplementation of the transport and storage subsystems of the
- Xnews software -- basically, everything except news readers. We supply a
- Xsimple news reader (written by Michael Rourke, included by permission,
- Xslightly modified [so bugs are probably our fault]) as a replacement for
- XB-News readnews suited to use by occasional users. For regular news users,
- Xthere are several more sophisticated readers widely available, and all should
- Xwork with C News. We use Larry Wall's "rn" ourselves; we have not included
- Xit because this distribution is already rather big.
- X
- XC News's major advantage over B News is that it is much faster. Timings
- Xquite a while ago gave C News a speed advantage of roughly a factor of 25
- Xin processing incoming batches. This has probably improved a bit since.
- XC News is now, on good machines with good C libraries, mostly system-call
- Xbound. Use of system calls has been optimized with some care, so it's
- Xunlikely that further big speed improvements can be made at user level
- Xwithout sacrificing backward compatibility. See our paper in the Winter '87
- XUsenix proceedings for some discussion of how the speed was achieved.
- X
- XC News also wins over B News on simplicity and robustness. We provide
- X(in our opinion) everything that's necessary, and avoid the frills that
- Xrun up the complexity and decrease reliability. We have not attempted to
- Xprovide every feature anyone can think of, and have no plans to do so
- Xin future either. (This is one reason why we've stayed out of the news-
- Xreader business, which generally has a bad case of feature-of-the-week
- Xsyndrome.)
- X
- XC News's files are fully compatible with those of B News for any program
- Xthat does not read log files and does not inspect the middle field of the
- Xhistory file closely. (The one major program that does is "nntp"; we include
- Xdiffs for NNTP 1.5 which handle our middle-field format, interface it to
- Xour input subsystem, and generally make it quite a bit faster than the
- Xoriginal.) C News complies fully with RFC 1036 (nee RFC 850), the official
- Xdefinition of news interchange format.
- X
- XC News is, by intent and *we think* in practice, compatible with B News
- Xat the level of most interfaces to the normal user, which basically means
- Xthe semantics and options of "inews". It is *not* compatible on the
- Xsystem-administration level, although we think most of the changes are
- Ximprovements or worthwhile simplifications. The "postnews" that we supply
- Xis not compatible with that of B News; it is purely interactive, as news
- Xthat is already formatted can simply be fed to "inews -h".
- X
- XFor those who now run one of our ancient pre-alpha versions, many things
- Xhave changed, and in particular the four-field history file format is gone.
- XC News has also changed quite a bit since the alpha release that went out
- Xon Usenet some time ago.
- X
- XFor our beta testers, build and the Makefiles have changed quite a bit but
- Xthe software itself needed only fairly trivial fixes.
- X
- XWe know of three things that could still use work in this release:
- X
- X 1. The documentation could use work, especially for naive customers.
- X As it stands, it pretty much assumes a general knowledge of news
- X software.
- X
- X 2. There are a great number of small improvements that could
- X be made to the installation process, especially to permit still more
- X customization via the "build" program.
- X
- X 3. The fgetmfs function (in the libc directory) assumes that fgets
- X does not alter the buffer beyond the end of the string. We are not
- X sure how portable this is, although it works on all our beta-test
- X systems, and may revise fgetmfs someday.
- X
- XThe active file format is the 4-field one that B news introduced midway
- Xthrough 2.10, with minor additions: an `x' in the 4th field means
- X``discard articles in this newsgroup'', and `=group' in the 4th field
- Xmeans ``file articles for this newsgroup under `group' instead''.
- X
- XThe history file format is like B with one exception: the second field,
- Xwhich few programs ever look at, now consists of two subfields separated
- Xby a tilde (~). The first is the arrival date as a decimal number, the
- Xsecond is the expiry date (if any) as a human-readable date (as emitted by
- Xrnews) or a decimal number (after expire has gotten its hands on it once).
- XExpire is tolerant of human-readable dates in both those places, but other
- Xthings may not be. The best way to get the history file into the new
- Xformat is to rebuild it completely (this is RELATIVELY quick).
- X
- XThe sys file format is like a late-model B news with two extensions.
- XFirst, the second field (groups and distributions) may optionally be
- Xsplit into two subfields (newsgroups and distributions, respectively)
- Xwith a slash. This permits solutions to various tricky problems that can
- Xarise in odd situations if it is impossible to tell what's a newsgroup
- Xname and what's a distribution. Second, there is a new flag in the
- Xthird field: f is like F except that its output has the size
- Xinformation that the C batcher wants for accurate limiting of batch
- Xsize.
- X
- XThe way the news articles themselves are stored is totally unchanged; we
- Xhave been unable to think of any changes that are worth the trouble.
- X
- XThere are some new control files in /usr/lib/news, to control the smarter
- Xexpire and batcher. (Old C News sites, note some format changes.)
- X
- XFile organization: the one change is that programs are now kept mostly
- Xin /usr/lib/newsbin, with /usr/lib/news reserved for control files etc.
- X
- XB News sites note: /bin/rnews is now a front end for the input spooler.
- XThe real news-filing program is called relaynews and is not in /bin.
- X
- XC News is meant to run adequately on a 16-bit machine, although this has
- Xnot been tested thoroughly since utzoo became a Sun.
- X
- XMost (by intent all) of the programs understand seven key environment
- Xvariables: NEWSARTS specifies location of articles (default
- X/usr/spool/news), NEWSCTL specifies location of control files (default
- X/usr/lib/news), NEWSBIN gives location of programs (default
- X/usr/lib/newsbin), NEWSUMASK gives the umask to be used in creating
- Xfiles (default 002), NEWSPATH gives the path used to find "normal" programs
- X(default /bin:/usr/bin), NEWSMASTER is the address to which problem
- Xreports should be mailed (default usenet), and NEWSCONFIG is the full
- Xpathname of a little file that shell programs can use to pick up local
- Xdefault settings for all these things (the equivalent for the C programs
- Xis in the C News library) (default /usr/lib/news/bin/config). The
- Xenvironment variables override the defaults for testing and for operation
- Xin funny situations. Note that one or two things (e.g. relaynews), as
- Xdistributed, will insist on renouncing setuid privileges if invoked with
- Xthese overrides.
- X
- XBe warned that the nntp stuff in nntpdiffs, and the simple news reader in
- Xrna, have not been gone over very well to make sure that they use the
- Xstandard configuration mechanisms. Hardwired pathnames may be present there,
- Xand in general the stuff there is not well fitted into our automatic-install
- Xmachinery.
- X
- XSee ROADMAP for a run-down on what's where in the distribution. See
- Xdoc/install for how to install it. Conf/build is the interactive setup
- Xprogram that does most of the work, or rather, sets up shell files which,
- Xwhen run, do most of the work. Even if your system is odd enough that
- Xyou don't want to run the shell files conf/build generates -- doit.bin and
- Xfriends -- as is, you are most strongly advised to run conf/build and use
- Xthe resulting shell files as guides, rather than trying to "wing it"
- Xyourself. Getting the library built correctly is *NOT* trivial, because
- Xwe try to cater to all 57000 different variants of Unix and there are a lot
- Xof decisions that have to be made. This is why we supply a 31KB shell
- Xprogram to generate the configure+compile+install instructions, rather than
- Xjust sending a complete pre-cooked recipe embodied in a Makefile: there are
- Xtoo many variables affecting how it should be done.
- X
- XC News has been tested pretty thoroughly. We're also thoroughly sick of
- Xit and make no promises that there will ever be another release. We may,
- Xrepeat *may*, provide updates via some appropriate newsgroup (currently
- Xthe best choice is "news.software.b", although there is some sentiment for
- Xfolding all the subgroups there into just "news.software"; we oppose
- Xcreation of "news.software.c" because we don't think there will be enough
- Xtraffic to justify a whole newsgroup).
- X
- XIf you've found a problem, we definitely do want to hear about it. But,
- Xwe *do not* want to see 2000 lines of diff listing! What we want to see
- Xis a concise human-readable description of what the problem is and how,
- Xif at all, you solved it. If we want the diff listing, we will ask.
- XSimilarly, we are interested in hearing about changes and improvements,
- Xbut want to see terse descriptions first.
- X
- XIf you want us to consider changes/fixes/etc, send them to us, don't just
- Xpost them to the net. We don't necessarily read all possibly-relevant
- Xgroups. Only postings from us are officially part of C News.
- X
- XTo send comments, complaints, problem reports, etc., do *not* mail to
- XGeoff or Henry personally, but to:
- X
- X c-news@utstat.toronto.edu
- Xaka c-news@utstat.utoronto.ca
- Xaka utzoo!utstat!c-news
- X
- XUtstat.toronto.edu is directly on the Internet but does not have a lot of
- XUUCP connections. However, it does talk to utzoo. Utzoo connects to half
- Xthe known universe (well, not quite, but try via allegra, att, attcan,
- Xattunix, decvax, dptcdc, floyd, hoptoad, kitty, linus, mnetor, pyramid,
- Xsuncan, utai, utgpu, watmath, or yunexus).
- X
- X Geoff Collyer
- X Henry Spencer
- !
- echo 'ROADMAP':
- sed 's/^X//' >'ROADMAP' <<'!'
- Xbatch Output batcher. It should work with B News, but not as well --
- X it really wants to be told sizes of articles as well as names.
- Xconf Configuration stuff, including the "build" shell program that
- X does most of the work of installing C News.
- Xcontrib Software from other contributors, possibly useful but not really
- X an official part of C News.
- Xdoc User documentation, including "install" which discusses how to
- X install C News.
- Xexpire Expire and friends, including history rebuilding and active-file
- X updating (neither of which is done by expire in C News). Ought
- X to work with B News. Fast, and permits control of expiry time
- X and such on a per-group basis.
- Xh Header files for include (see below).
- Xhfake Header files that your system ought to have but possibly doesn't,
- X for include (see below).
- Xinclude Actually not in distribution -- "build" constructs this from h and
- X hfake as required.
- Xinput Input spooler and processing. Includes the "rnews" that goes in
- X /bin or wherever.
- Xlibsmall
- Xlibbig Libraries for a couple of key functions that care whether they
- X have a large address space to work with.
- Xlibbsd42
- Xlibusg
- Xlibv7
- Xlibv8 Libraries for details that are Unix-variety-specific.
- Xlibc Stuff that ought to be in your C library but probably isn't.
- Xlibcnews General C News library functions.
- Xlibfake Stuff that might be in your C library, but might not be.
- Xlibstdio Fast re-implementations of some crucial stdio functions.
- X These are compatible with most Bell-derived stdios, and quite a
- X bit faster than most of the pre-System-V ones (which includes
- X those of many 4BSD variants).
- Xman Manual pages.
- Xmisc Miscellaneous internal utilities and maintenance programs.
- Xnntpdiffs Amendments to NNTP version 1.5, for Internet users, to work
- X with C News history-file format and to be a good deal faster.
- Xnotebook The C News Implementor's Notebook -- pieces of documentation
- X aimed more at gurus than novices.
- Xrelay Relaynews and friends. The heart of C News -- actual reception
- X and filing of articles.
- Xrna A simple version of readnews (by Michael Rourke at UNSW) for
- X naive news users.
- !
- echo 'doc/install':
- sed 's/^X//' >'doc/install' <<'!'
- X.de Fn
- X\\&\\$1\\fB\\$2\\fP\\$3
- X..
- X.TL
- XInstalling ``C News'' Network News Software
- X.AU
- XGeoff Collyer
- X.AI
- XDepartment of Statistics
- XUniversity of Toronto
- X.AB
- XThis document describes the flow
- Xof network news within and between machines,
- Xwhat each component of C news does,
- Xand
- Xhow to install C news.
- X.AE
- X.SH
- XIntroduction
- X.PP
- XNetwork news
- X(or
- X.I netnews
- Xfor short)
- Xconsists of a collection of messages formatted similarly to
- XARPAnet mail
- X(see ARPA Internet RFC 1036 for details),
- Xwidely spread.
- XThe logical network,
- Ximposed on top of various real networks,
- Xformed by the set of all interconnected sites
- Xexchanging network news is called ``Usenet''
- Xand was formed in 1979,
- Xradiating out from Duke University.
- XNetnews is propagated
- Xbetween cooperating machines
- Xby a flooding algorithm,
- Xwith some loop prevention heuristics:
- Xeach machine sends its neighbours news that the
- Xneighbours have (probably) not yet seen.
- X.PP
- XFlow of netnews between machines
- Xmay be achieved by use of any network
- Xwhich can transmit an arbitrary stream of
- X(at least 7-bit)
- XASCII code, unmodified.
- XIf a network cannot transmit ASCII intact
- X(e.g. BITNET),
- Xit is possible to encode netnews
- Xbefore transmission across the network
- Xand
- Xdecode it upon reception from the network.
- XSince one cannot be certain that
- Xone's network neighbours will be up and
- Xreachable at all times,
- Xoutgoing netnews must be queued,
- Xat least in the case of network trouble.
- XTo date,
- Xat least
- Xthese networks,
- Xprotocols and media
- Xhave been used to transmit netnews correctly:
- XUUCP,
- XRS232,
- XNNTP,
- XEthernet\(rg,
- Xthe ARPA Internet,
- XDatakit\(rg,
- XACSnet,
- Xmagnetic tape,
- XSMTP,
- Xand
- XBITNET,
- Xthough at least the last two require some form of encapsulation
- Xto avoid corruption of articles;
- XSMTP because some common implementations get the newline-CRLF
- Xmappings wrong,
- Xthus throwing off byte counts in batches,
- Xand
- XBITNET because of its Procrustean chopping,
- Xexpanding,
- Xmapping,
- Xbashing
- Xand
- Xsmashing
- Xof all data sent through it
- X(sending lines of 80 or fewer characters of
- Xletters of either case and digits and plus and minus
- Xappears to be safe).
- X.PP
- XNetnews arrives via some network,
- Xwhich causes a command to be executed upon arrival
- X(e.g.
- X.I rnews ).
- X.I rnews
- Xtypically spools the inbound netnews for later processing.
- XEventually
- X(often within the hour
- Xor at the end of the business day),
- Xthe input queue is run
- Xand the netnews is stored locally,
- Xtypically under
- X.I /usr/spool/news ,
- Xand queued for transmission to netnews neighbours.
- XOnce stored on disk,
- Xnetnews may be read by any of a collection of unprivileged news readers,
- Xincluding
- X.I cat (1).
- X.I Expire
- Xis run typically each night
- Xto remove old netnews from disk.
- X.PP
- XC News was originally written to provide a
- Xmuch faster and smaller,
- Xmore robust,
- Xreliable
- Xand
- Xcorrect
- Ximplementation of netnews software than B 2.11 news.
- XWe believe that C News is also
- Xfaster,
- Xsmaller
- Xand
- Xmore robust
- Xthan B 3.0 news.
- XB 3.0 news has many more features;
- Xtake that as you will.
- X.SH
- XC News Components.
- X.PP
- X.I Rnews
- Xinvokes
- Xthe input subsystem,
- Xwhich
- Xspools incoming netnews in its original form,
- Xas received,
- Xtypically in compressed batches.
- XPeriodically,
- Xthe input queue should be run
- Xby invoking
- X.I newsrun ,
- Xthus uncompressing any compressed input
- Xand feeding it to
- X.I relaynews .
- X.PP
- X.I Relaynews
- Xfiles incoming netnews as articles on disk
- Xand
- Xinitiates spooled transmission to other machines,
- Xoften by simply writing the names of the disk files
- Xcontaining the articles on the ends of `batch' files of file names.
- XQuite a bit of policy from RFC 1036 is embedded in
- X.I relaynews .
- X.I inews
- Xis a complex front-end for
- X.I relaynews
- Xwhich implements much of the per-site policy on news posting.
- X.I inews
- Xis a fairly-slow shell script which is adequately fast for most sites,
- Xbut it will need to be replaced by something more streamlined
- Xfor sites which gateway mailing lists into netnews,
- Xfor example.
- X.PP
- XThe output
- X.I batcher
- Xreads lists of file names
- Xand
- Xgenerates news batches
- X(see RFC 1036 or
- X.I news (5)
- Xfor the format)
- Xof prescribed sizes
- Xand queues the batches
- Xfor transmission to other sites.
- XThe batcher is run asynchronously with
- X.I relaynews ,
- Xtypically once an hour
- Xoutside of business hours.
- X.PP
- X.I Expire
- Xis generally run once per night
- Xto remove from disk news articles older than a few days.
- X.I Expire
- Xcan use different expiry criterion for different newsgroups
- Xand can archive articles instead of removing them.
- X.I Expire
- Xalso runs asynchronously
- Xwith
- X.I relaynews .
- X.PP
- XThere are many news readers.
- XC News
- Xcomes with a limited news reader
- X(\c
- X.I readnews
- Xby Michael Rourke)
- Xsufficient to replace
- Xlong-winded
- X.I /etc/motd s
- Xbut you will want a heavy-duty news reader
- Xif you plan to read more than local news groups.
- XWe recommend
- X.I rn
- Xby Larry Wall,
- Xavailable from
- Xyour netnews neighbours
- Xor
- Xyour nearby
- X.B comp.sources.unix
- Xarchive site.
- XThere are others:
- X.I vn
- Xand
- X.I vnews
- Xare two.
- X.SH
- XPreparation for Installation
- X.PP
- XNetnews consumes a lot of disk space
- Xand
- Xoften a lot of transmission time.
- XHere are some relevant measurements regarding a full netnews feed
- Xas of the time of writing
- X(January 1989),
- Xtaken from
- X.I news.lists :
- Xa day's netnews is about 3Mb and 1,400 articles
- Xin 450 newsgroups.
- XYears ago,
- Xsites often kept 14 days of netnews on disk,
- Xbut now many sites keep news for 3 to 5 days,
- Xthus allowing for the occasional long weekend.
- XThus a full news feed expired after 4 days will consume
- Xabout 12Mb.
- XSome people feel that news volume is doubling roughly every 16 months.
- XIf this is true,
- Xwe can expect volume to increase by a factor of 10
- Xin about 4 years
- Xto 30Mb per day or 115Mb for 4 days.
- XIt is thus wise planning to set aside a lot of disk space for netnews.
- XThere are two ways to cope with ever-increasing volumes of netnews:
- Xrefuse to accept more newsgroups,
- Xor
- Xexpire news after shorter intervals on disk.
- XA current full feed takes just over 7 hours to transmit at 1200 baud,
- Xso for dial-up connections
- Xone clearly wants the fastest modems one can afford.
- X.PP
- XClearly,
- Xtransmitting a full news feed is a non-trivial commitment of resources,
- Xso you may have some difficulty in finding a site willing to supply one.
- XSuch a site may in turn expect you to feed yet other sites.
- XYou will need to agree with your prospective netnews neighbour(s)
- Xupon transfer media,
- Xprotocols
- Xand
- Xnetworks.
- X.PP
- XBefore proceeding to install C News, you should read this document through
- Xto the end, probably read the companion document
- X\fIThe Interface Between C News And The Outside World\fR,
- Xand possibly read selected items in the
- X\fIC News Implementor's Notebook\fR and the manual pages.
- X.PP
- XYou will need to
- Xassign a user id and group id to netnews
- X(often new ids called ``news'');
- Xinitialise these directories:
- XNEWSCTL
- X(typically
- X.B /usr/lib/news ),
- XNEWSBIN
- X(\c
- X.B /usr/lib/newsbin ),
- Xand
- XNEWSARTS
- X(\c
- X.B /usr/spool/news );
- Xand
- Xinstall each subsystem.
- XNEWSCTL and NEWSARTS
- Xare logically one subtree,
- Xdefining a news data base,
- Xbut are split for backward compatibility with
- Xolder news software,
- Xparticularly old news readers.
- XNEWSBIN
- Xcontains programs and shell scripts
- Xwhich might be common amongst machines sharing
- Xa common architecture
- X(e.g. Sun 3's);
- X.Fn NEWSCTL /bin
- Xmay override these.
- XThe goal is to install the subsystems,
- Xintegrate them into a working news system,
- Xand
- Xconfigure the news system to communicate with other news systems.
- X.PP
- XThere are a few key files that must exist before any serious
- Xattempt may be made to operate the news software.
- X.Fn NEWSCTL /active
- Xis the list of newsgroups that this site knows
- X(is willing to accept or individually reject),
- Xand must be owned by the
- XNEWS
- Xuserid
- X(the userid that owns
- X.Fn NEWSBIN /relay/relaynews ,
- Xtypically
- X.I news ).
- XYou will probably want to get your initial
- X.Fn "" active
- Xfile from your upstream feed
- Xand edit it to suit the set of groups you wish to receive.
- XBe sure to make the second field more than five digits wide,
- Xby adding leading zeroes
- X(ten digits is a conservative width).
- X.Fn NEWSCTL /sys
- Xdefines the newsgroups that this site is willing to accept
- Xand describes how they are to be forwarded to its neighbours.
- X.Fn NEWSCTL /server
- Xcontains the hostname of your file server,
- Xif you have multiple machines sharing news
- Xvia a network file system.
- X.Fn NEWSCTL /whoami
- Xsimilarly contains the name by which a cluster of machines
- Xsharing news is to be known for purposes of news.
- X.Fn NEWSCTL /mailname
- Xis optional and contains the full
- X(possibly dotted)
- Xname by which your cluster is known for purposes
- Xof mail.
- X.Fn NEWSCTL /organisation
- X(or
- X.Fn NEWSCTL /organization
- Xif you insist)
- Xcontains the name of your organisation,
- Xwhich will be copied into the
- X.B Organization:
- Xheader of articles posted locally,
- Xby default.
- X.Fn NEWSCTL /mailpaths
- Xdefines mail routes to machines which contain aliases
- Xfor postings to moderated newsgroups.
- X.Fn NEWSCTL /log ,
- X.Fn NEWSCTL /errlog ,
- X.Fn NEWSCTL /history ,
- X.Fn NEWSCTL /history.dir ,
- Xand
- X.Fn NEWSCTL /history.pag
- Xmust exist and be owned by the
- XNEWS
- Xuserid.
- XTentative versions of all these files are built by the installation
- Xprocedures, but it is quite likely that you'll have to edit some of them.
- X.SH
- XC News Installation
- X.PP
- XProceed to the
- X.Fn \& conf
- Xdirectory of the distribution.
- XThere is a major shell file here, named
- X.Fn \& build ,
- Xthat will interrogate
- Xyou at length and construct shell files to do the real work.
- XYou may need to do
- X``chmod\ +x\ build'' before running it, to make it executable.
- X.PP
- XYou will probably need your system's manuals on hand to answer
- X.Fn \& build 's
- Xquestions.
- XAnother terminal (or another window, on a bitmap display) would also be
- Xuseful.
- XYou'd best be prepared to take notes, also, as
- X.Fn \& build
- Xwill occasionally suggest that something be checked when you're done.
- X.PP
- X.Fn \& Build
- Xitself does not alter any files or perform any installation
- Xchores: all it does is create shell files in the
- X.Fn \& conf
- Xdirectory.
- XIf you already know something about news software, or are merely
- Xsuspicious that
- X.Fn \& build
- Xhasn't done everything right, you should
- Xprobably read the shell files before running them.
- XThere are four of them:
- X.Fn \& doit.root ,
- X.Fn \& doit.bin ,
- X.Fn \& doit.news ,
- Xand
- X.Fn \& again.root .
- X.PP
- X.Fn \& Doit.root
- Xsets up the major directories for news, and sets their
- Xownerships correctly.
- XIt typically will have to be run as \fIroot\fR to have adequate permissions
- Xfor all of this.
- XIt is brief.
- X.PP
- X.Fn \& Doit.bin
- Xdoes most of the work of building and installing the programs.
- XIt should be run as the user who owns the distribution directories and will
- Xown the executable programs under NEWSBIN.
- X.PP
- X.Fn \& Doit.news
- Xdoes some other small chores and installs control files.
- XIf any of the control files already exist, it will complain and refuse
- Xto overwrite them, as a safety precaution.
- XIt should be run as the owner of the news files.
- XSince many of the files it is installing are built by
- X.Fn \& doit.bin ,
- Xthat should be run first.
- X.PP
- XFinally,
- X.Fn \& again.root
- Xtends to ownership and permission changes on
- Xa few programs that need to run set-userid.
- XIt requires the ability to change ownerships and to set permissions on
- Xthe files afterwards, which usually means running it as \fIroot\fR.
- XIt too is brief.
- X.PP
- XThere are undoubtedly strange systems out there that
- X.Fn \& build
- Xand friends are not smart enough to cope with.
- XIn such cases it will be necessary to edit the shell files before running
- Xthem, or to use them as guides and do the work by hand.
- XIn particular, systems that require strange options in
- X.Fn \& Makefile s
- Xwill need to have those inserted by hand.
- X.PP
- XIf you want to test pieces of C News without installing them, some
- X(not all) of the subsystems have a ``make\ r'' feature that runs a
- Xregression test.
- XNote: almost all of these require that
- X.Fn NEWSCTL /bin/config ,
- Xor its local equivalent, be in place already so that shell files
- Xcan find out what PATH (etc.) they should be using.
- X.PP
- XNote that it is easy to build a test news system which is completely
- Xindependent of other existing news systems on the same machine,
- Xor
- Xto build one which shares its
- X.Fn NEWSBIN ""
- Xwith another C news system,
- Xor shares input of articles
- X(e.g. running an old B news system
- Xand a new C news system off the same stream of input
- Xuntil you are confident that your new C news system is operating
- Xto your satisfaction).
- XSee
- X.I subst (1)
- Xfor the mechanism which permits quickly changing
- Xall the places that know the location of
- X.Fn NEWSCTL /bin/config.
- XAfter that,
- Xedit this news system's
- X.Fn NEWSCTL /bin/config
- Xand things should be set up for separate existence.
- X.SH
- XFirst News
- X.PP
- XWhen you arrange to get a news feed from your neighbor, you should also
- Xask him to send you the current set of articles in the newsgroup
- X\fBnews.announce.newusers\fR.
- XSeveral of these are very important reading for people who are new to the net.
- X.SH
- XUnusual Systems
- X.PP
- XWe believe that C News runs fine on 16-bit machines, but it hasn't been
- Xtested very thoroughly on them lately.
- XIt will not perform quite as well with the more limited space.
- X.PP
- XMachines with very old compilers can be a headache.
- XThere are some hooks in
- X.Fn \& h/news.h
- Xfor doing without ``void'' and ``unsigned long'',
- Xtwo particular problem areas, but they have to be arranged manually;
- X.Fn \& build
- Xdoes not know about them.
- X.PP
- XSome very old systems cannot do \fIsetuid(geteuid())\fR, which makes it
- Ximpossible for a daemon to make directories and get the ownership right.
- XWe provide a small program,
- X.Fn \& setnewsids ,
- Xto run setuid-root.
- X.Fn \& Relaynews
- Xknows about it and invokes it if \fIsetuid(geteuid())\fR fails;
- Xit then sets permissions correctly and re-invokes
- X.Fn \& relaynews .
- XThe code is short enough to be read and understood in full, so that the
- Xsuspicious system administrator can be sure that this setuid-root program
- Xis not going to do something awful.
- !
- echo 'doc/README':
- sed 's/^X//' >'doc/README' <<'!'
- XThis is user documentation. See ../notebook for implementor's notes.
- X
- XEverything here is set up to use the -ms macros. No complex formatting
- Xis attempted, and only the three standard fonts (R, I, B) are used.
- X
- Xinstall somewhat sketchy how-to-install document
- Xinstall.out nroffed version of install
- Xinterface technical summary of how C News works and its interface to
- X the rest of the system
- !
- echo 'doc/interface':
- sed 's/^X//' >'doc/interface' <<'!'
- X.DA "9 June 1989"
- X.TL
- XThe Interface Between C News And The Outside World
- X.AU
- XHenry Spencer
- X.AI
- XDept. of Zoology
- XUniversity of Toronto
- X.SH
- XIntro and Generalities
- X.PP
- XC News relates to the ``outside world'',
- Xthe system it is installed on, in a number of ways.
- XThis document attempts to enumerate them and explain what goes on.
- X.PP
- XIn general, C News attempts to rely only on ``common basic Unix'',
- Xand in particular it is not particularly BSD-specific or System-V-specific.
- XSpecifically,
- Xit makes no use of ornate locking mechanisms, silly interprocess-communication
- Xschemes, peculiar networking primitives,
- Xextensions to C, or other annoyances.
- X.SH
- XDirectories And Userids
- X.PP
- XMost of the components of C News live in \fI/usr/lib/newsbin\fR,
- Xwith control files in \fI/usr/lib/news\fR and spooling areas
- X(for current news, inbound traffic, and outbound traffic) in
- X\fI/usr/spool/news\fR.
- XSee the document
- X\fIDirectory Layout and PATH in C News\fR
- Xfor elaboration on the structure,
- X\fIFiles in /usr/lib/news (aka NEWSCTL)\fR for a summary of control files,
- Xand
- X\fIConfiguration Mechanisms in C News\fR
- Xfor how to change the locations of the directories.
- X.PP
- XThere is an extensive subdirectory structure under \fI/usr/lib/newsbin\fR,
- Xwith only a few heavily-used utilities in the top-level directory.
- XIn the following,
- Xprograms not explicitly described as going elsewhere are all under there
- Xsomewhere.
- X.PP
- XC News's spool areas and major control files need to be owned by a
- Xspecific userid, normally \fInews\fR.
- X(We believe that nothing except some of the Makefiles knows this name.)
- XWe suggest that this userid should not own anything else on the system.
- XThe programs in \fI/usr/lib/newsbin\fR can be
- Xowned by \fIbin\fR (or anyone else) except for those that need to be setuid.
- X(On our systems the non-setuid ones are owned by \fIbin\fR.)
- X.SH
- XUnix Utilities
- X.PP
- XC News requires a fairly complete Unix or equivalent.
- X(We take no position on whether 4BSD, or System V, is Unix or not;
- Xour private opinion is that neither truly deserves the name any more,
- Xalthough we occasionally change our minds about which is less deserving.)
- X(Note also that ``Unix'' is used here as an abbreviation for
- X\fBThe UNIX Operating System\fR\(rg,
- Xa registered trademark of AT&T;
- Xwe acquired our bad habits and incorrect capitalization from
- XUnix [sic] documentation supplied by the Bell System in the mid-1970s.)
- X.PP
- XIn particular, C News relies heavily on shell files.
- X``Shell'' here means, of course, the standard shell, written by
- XSteve Bourne.
- XIf your \fI/bin/sh\fR is not a Bourne shell or \fIvery\fR good imitation,
- Xyou're in trouble.
- XNote, in particular, that some old versions of the Korn shell differ from
- Xthe Bourne shell in some important details of quoting behavior, and
- Xwe \fIknow\fR this causes problems with C News.
- XYou need a standard shell.
- X.PP
- XTo the best of our ability, all our shell files begin with ``#!\ /bin/sh''.
- XIf your shell misinterprets this as a request to run the C shell, you're
- Xin trouble.
- XIf your shell doesn't know about ``#'' comments at all, you're in trouble.
- XWe hope that no modern shell makes either of these mistakes.
- X.PP
- XWe believe that none of our stuff relies on relatively modern shell features
- Xlike shell functions or \fIgetopts\fR (as distinct from \fIgetopt\fR).
- XIf \fItest\fR and \fIecho\fR are not built-in commands in your shell,
- Xefficiency will suffer
- Xbut everything should still run.
- XIt's possible that a few obscure things will break if your shell does not
- Xsupport ``['' as a synonym for \fItest\fR, although we try to avoid this usage.
- X.PP
- XWithin the shell files, C News makes heavy use of a wide variety of Unix
- Xutilities, notably \fIsed\fR and \fIawk\fR.
- XAny \fIawk\fR should do; in particular, nothing needs the ``new \fIawk\fR''
- X(we don't run it yet).
- XAlthough we have tried to avoid it, it is possible that some things depend
- Xon \fIawk\fR recognizing ``\et'' inside strings, which very old \fIawk\fRs
- Xdidn't.
- X.PP
- XIf your Unix does not put all standard Unix programs in
- Xthe standard directories\(em\fI/bin\fR and
- X\fI/usr/bin\fR\(emyou will need to modify C News's default PATH to include
- Xwhatever bizarre directories are involved.
- XSee the \fIConfiguration Mechanisms\fR document for details.
- XIn particular, for some insane reason, 4BSD relocated a few standard
- Xutilities like \fIwc\fR to \fI/usr/ucb\fR, which causes trouble
- X(and not just to C News!).
- XWe simply put some links in \fI/usr/bin\fR to fix this on our systems,
- Xand this is what we recommend you do, but if you can't, you'll have to
- Xchange the PATH.
- X.PP
- XSeveral parts of C News rely on being able to send mail to an administrative
- Xaccount (the name of which can be chosen; see \fIConfiguration Mechanisms\fR).
- XThe assumption is that a message, without any RFC822 headers, can be piped
- Xinto \fI/bin/mail\fR (or whatever \fImail\fR is first in the PATH)
- Xinvoked with a single argument which is the addressee,
- Xand be delivered to the addressee's mailbox intact, possibly embellished
- Xwith headers.
- XThat is, with news's standard PATH, if
- X.DS
- Xecho hi there Joe | mail joe
- X.DE
- Xdoes not result in the message ``hi there Joe'' arriving in the mailbox
- X``joe'', you're going to have to fake it.
- XSee \fIDirectory Layout\fR for insight on where to put a fake \fImail\fR
- Xso that C News components will use it rather than \fI/bin/mail\fR.
- X.PP
- XSimilarly, several parts of C News rely on being able to invoke \fIhostname\fR
- Xwithout arguments, and have it return a single word which is the name of the
- XCPU the code is running on.
- XAgain, you'll have to fake it if you don't have \fIhostname\fR
- Xor if it outputs something strange.
- X.PP
- XInput reception and output batching both want to use the \fIcompress\fR
- Xdata-compression program.
- X\fICompress\fR has been published on Usenet and is shipped with many Unix
- Xsystems these days, but if you don't have it, we recommend that you get
- Xit from your neighbors or the \fIcomp.sources.unix\fR archives.
- XIt is possible to configure C News so that it doesn't use \fIcompress\fR
- Xto compress news being transmitted, but you don't want to.
- X\fICompress\fR is good and it is fast.
- X.SH
- XLibraries
- X.PP
- XC News is mostly self-contained as far as libraries go.
- XIt does need some things that are not yet universal, like a full set
- Xof string functions (e.g. \fIstrtok\fR);
- Xwe provide reasonably portable versions of these for places that lack them.
- X.PP
- XOne thing that C News needs, because it is both useful
- Xand a user-visible part of B News,
- Xis the \fIdbm\fR library.
- XAT&T has stupidly omitted it from System V.
- XWe include an emulation or two that are claimed to work reasonably well.
- XIf your system doesn't have a real \fIdbm\fR, however, our recommendation
- Xis that you harass your supplier about it.
- X.SH
- XNetworking
- X.PP
- X(We're talking here about networking in the sense of NFS and all those fun
- Xthings, not \fIuucp\fR or TCP/IP.)
- XC News is designed to run on systems which consist of a conglomeration of
- Xmachines on a local network, with only one of them acting as news server.
- XBe warned, though, that if you're doing this, you need to have \fIrsh\fR
- Xor some reasonably equivalent way of executing a program on another CPU.
- XIf you've only got one machine,
- Xjust make sure you \fIdon't\fR have a \fI/usr/lib/news/server\fR file and
- Xforget the whole issue.
- X.SH
- XHighly System-Specific Things
- X.PP
- XThere are two small utilities within C News that are inevitably highly
- Xsystem-specific and have a high probability of needing changes to match
- Xyour system.
- XBoth normally live in \fI/usr/lib/newsbin\fR proper.
- X.PP
- X\fISpacefor\fR is invoked by various components of C News to find out
- Xhow much disk space is available.
- XThe space margins in it are ones we find reasonable, but you may need
- Xto adjust them.
- XOn an old System V system in particular (one where \fIdf\fR can't be applied
- Xto any directory name, but needs to be given a name in \fI/dev\fR), you
- Xwill also probably need to modify the locations to be \fIdf\fRed.
- XYou will also need to fix \fIspacefor\fR if your system's \fIdf\fR yields
- Xresults in some strange format or in some strange units.
- XWe believe that we get it right for stock 4BSD,
- Xmany (but probably not all) System Vs, modern Irix, and real Unix (Version 7).
- XBeware introducing major inefficiencies:
- X\fIspacefor\fR is called a lot.
- X(Our current one could stand to be faster, in fact.)
- XBeware, also, that \fIspacefor\fR is not intended to permit routine operation
- Xusing filesystems that are on the brink of being full;
- Xthe limits it imposes are only approximate, and no attempt has been made
- Xto tune its clients to exploit every last available block.
- X.PP
- X\fIQueuelen\fR is invoked by the batcher to determine how long the current
- X\fIuucp\fR queues are, so it can judge whether it should suspend batching
- Xof output to a given site.
- XThere is too much diversity in \fIuucp\fRs for us to try to do this for all
- Xpossible versions.
- XWe think we get it right for the two most common flavors
- X(HDB, aka BNU, and old subdirectory versions).
- XOur versions measure queue length in batches, not bytes,
- Xbut it would not be hard to change this:
- X\fIqueuelen\fR, the \fIbatchparms\fR control file, and the \fIsendbatches\fR
- Xhow-many-batches-should-I-try-to-add logic need to agree on the units of
- Xmeasurement, but (we think) nothing else cares.
- X.PP
- XIt is just possible that you might have to modify \fInewshostname\fR,
- Xwhich also lives in \fI/usr/lib/newsbin\fR itself.
- X\fINewshostname\fR delivers the name which should be applied to the
- Xwhole system (not just the particular CPU) for news purposes.
- XIt tries to be fairly clever about different ways of getting the name,
- Xand in particular will take it out of \fI/usr/lib/news/whoami\fR if
- Xthat file exists, but if you're doing something odd on a strange system,
- Xchanges may be needed.
- X.SH
- XInput
- X.PP
- XInput from the net via \fIuucp\fR (or equivalent) shows up as batches of
- Xnews to be fed to \fIrnews\fR or its obsolete synonym \fIcunbatch\fR.
- XThese two programs normally live in \fI/bin\fR, although nothing in the
- Xcode knows about this and they can go elsewhere (one of our systems does this).
- XBoth are simple front ends that invoke \fIinput/newsspool\fR to store the
- Xbatch, while taking precautions to report trouble and not to overflow disks.
- XNeither \fIrnews\fR nor \fIcunbatch\fR needs to be setuid.
- X.PP
- XInput via NNTP over the Internet (or equivalent) uses rather different
- Xmachinery but ends up creating a saved batch in much the same way as
- X\fIinput/newsspool\fR does.
- X.PP
- X\fIInput/newsspool\fR is a small C program that saves a batch,
- Xwriting into a file in \fI/usr/spool/news/in.coming\fR.
- XIt must be able to create files there, and \fIinput/newsrun\fR (see below)
- Xneeds to be able to read them and delete them.
- XThis gets a little tricky because \fInewsspool\fR will usually be
- Xrun by \fIuuxqt\fR as userid \fIuucp\fR (or something like that),
- Xnot as \fInews\fR, which \fInewsrun\fR needs to run as.
- XThe recommended solution is to have
- X\fInewsspool\fR owned by \fInews\fR and setuid.
- XAn alternative is to give the \fIin.coming\fR directory
- Xthe userid of \fInews\fR and the groupid of \fIuucp\fR, or vice versa,
- Xand set permissions so that either can access it.
- XOne of our systems ran that way for a while.
- X.PP
- XTo actually process incoming news, \fIinput/newsrun\fR gets invoked
- Xto decompress the spooled batches and
- Xfeed them to \fIrelay/relaynews\fR (see below).
- XThere is an option for \fInewsspool\fR to invoke
- X\fInewsrun\fR when a batch is spooled,
- Xbut a (usually)
- Xpreferable method is to have \fIcron\fR invoke \fInewsrun\fR once an hour.
- X\fINewsrun\fR does its own locking to prevent multiple occurrences running
- Xsimultaneously.
- XThere is a related program, \fIinput/newsrunning\fR, that can be used
- Xto set or clear a flag that stops \fInewsrun\fR;
- Xthis may be a useful tactic if \fInewsrun\fR should not run at
- Xcertain times.
- XBoth \fInewsrun\fR and \fInewsrunning\fR must be run as \fInews\fR.
- X.PP
- XWhen a user posts news, he (or his news reader) does it by feeding the
- Xarticle to \fI/bin/inews\fR.
- XIn C News, \fIinews\fR is a complex shell file that attends to preliminaries
- Xand then invokes \fIrelay/relaynews\fR.
- X\fIInews\fR does not need to be setuid (indeed, we make no use of setuid
- Xshell files at all, since they are grossly insecure).
- X\fIRelaynews\fR is the heart of C News, the program that actually pulls
- Xbatches apart and places articles into the database.
- XIf users are to be able to post news, \fIrelaynews\fR should be owned
- Xby \fInews\fR and setuid.
- X.SH
- XNews Readers
- X.PP
- XC News is fully compatible with B News to any news-reader program that
- Xdoes not inspect the middle field of \fI/usr/lib/news/history\fR too closely.
- XStandard B News news readers work fine.
- XWe supply a simple news reader
- X(written by, and included with permission of, Michael Rourke)
- Xas a naive-user replacement for the B News
- X\fIreadnews\fR.
- XMore complex programs are preferable for serious news enthusiasts.
- XWe recommend Larry Wall's \fIrn\fR
- X(which we use, unmodified),
- Xbut there are others.
- X.SH
- XOutput
- X.PP
- X\fIRelay/relaynews\fR normally queues up news for transmission to other
- Xsystems by appending article names and sizes to batch files
- Xin subdirectories under \fI/usr/spool/news/out.going\fR.
- XThese are then processed by \fIbatch/sendbatches\fR, which should be run
- Xregularly, as \fInews\fR, by \fIcron\fR.
- X\fISendbatches\fR can be configured to use a variety of transmission
- Xmechanisms, the usual being \fIuux\fR.
- X.SH
- XExpiry And Related
- X.PP
- XNews articles are removed, possibly with archiving to an archive area,
- Xby the expiry subsystem.
- X\fIExpire/doexpire\fR should be invoked now and then,
- Xas \fInews\fR, by \fIcron\fR.
- XWe suggest nightly.
- X\fIDoexpire\fR actually invokes \fIexpire/expire\fR to do the dirty work.
- X.PP
- XC News \fIexpire\fR does not have an option to rebuild the
- X\fI/usr/lib/news/history\fR file from scratch,
- Xsince that has nothing to do with expiry.
- XTo rebuild \fIhistory\fR,
- Xe.g. if it has been destroyed,
- Xuse \fIexpire/mkhistory\fR.
- X.PP
- XSome news readers need to have the third field of \fI/usr/lib/news/active\fR
- Xupdated occasionally to show the lowest article number still present in each
- Xnewsgroup.
- XFrankly, we think such news readers simply need to be fixed.
- XC News \fIexpire\fR does not do this updating.
- XFor those who use such news readers, however, \fIexpire/upact\fR will do
- Xsuch an update.
- XIt should be run as \fInews\fR.
- X.PP
- X\fIRelay/relaynews\fR
- Xdoes not implement the ``Supersedes:'' header, which we loathe
- Xas a crude solution to the wrong problem.
- XAlas, the network-map distribution in
- X\fIcomp.mail.maps\fR relies heavily on it.
- XOur preferred approach is to process map articles as they arrive and
- Xthen expire them normally (using \fIexpire\fR's expiry-date-override
- Xfeatures to insist that they expire promptly).
- XHowever, for those who don't do this, and don't want to have megabytes
- Xof obsolete map data piling up,
- X\fIexpire/superkludge\fR will remove superseded files in specified
- Xnewsgroups.
- XIt should be run as \fInews\fR by \fIcron\fR, perhaps nightly.
- X.SH
- XReboots and Administration
- X.PP
- XIf the system crashes, things like locks must be cleaned out if C News
- Xis to function properly after reboot.
- X\fI/etc/rc\fR, or equivalent, should run \fImaint/newsboot\fR during reboot,
- Xas \fInews\fR.
- X.PP
- XCertain log files can grow without bounds if not renamed/removed now and
- Xthen.
- XWe recommend running \fImaint/newsdaily\fR once a day.
- XIt tends to logs, keeping a generation or two for use in trouble tracking,
- Xand also sends mail to the news administrator in the event that something
- Xfunny has happened.
- X.PP
- XIn general, C News does not attempt to break locks,
- Xon the philosophy that a stale lock may mean something is badly wrong.
- X(See
- X\fILocking in C News\fR for details on locking methods.)
- XThe various programs will either give up, to return later, or wait
- Xpatiently for the lock to go away.
- XIf one doesn't keep an eye on things, a problem of some kind can hang up
- Xthe news system for quite a while.
- XRunning \fImaint/newswatch\fR once in a while\(emwe recommend a few times
- Xa day\(emwill alert administrators to signs of trouble.
- XExcept on grossly slow systems, C News locks should never hang around for
- Xany great length of time.
- !
- echo 'doc/Makefile':
- sed 's/^X//' >'doc/Makefile' <<'!'
- Xinstall.out: install
- X nroff -Tdumb -ms install >$@
- !
- echo done
-
-
-